Counterfeit detection scheme using paper surfaces and mobile cameras

ABSTRACT

Various authentication systems may benefit from detection of counterfeits. More particularly, certain authentication systems may benefit from a counterfeit detection scheme applicable to paper surfaces that can employ mobile cameras, such as the cameras associated with mobile phones. A method, according to certain embodiments, can include illuminating a surface of an item with a lighting source of a device. The method can also include capturing a plurality of images of the surface by a camera of the device during the illumination of the surface. The method can further include authenticating the item based on the plurality of images.

CROSS-REFERENCE TO RELATED APPLICATION

This application is related to and claims the benefit and priority ofU.S. Provisional Patent Application No. 62/255,134, filed Nov. 13, 2015,the entirety of which is hereby incorporated herein by reference.“Counterfeit detection using paper PUF and mobile cameras,” a paperpublished by the inventors in IEEE International Workshop on InformationForensics and Security (WIFS'15), Rome, Italy, 16-19 Nov. 2015, is alsoincorporated herein by reference in its entirety. “Counterfeit DetectionBased on Unclonable Feature of Paper Using Mobile Camera,” a paper bythe inventors for IEEE Transactions on Information Forensics andSecurity (TIFS), is also incorporated herein by reference in itsentirety.

BACKGROUND Field

Various authentication systems may benefit from detection ofcounterfeits. More particularly, certain authentication systems maybenefit from a counterfeit detection scheme applicable to paper surfacesthat can employ mobile cameras, such as the cameras associated withmobile phones.

Description of the Related Art

Merchandise packaging and valuable documents such as tickets and IDs arecommon targets for counterfeiters. Traditional high cost surfacestructures have sometimes been employed to defeat counterfeiting, suchas holograms, ultraviolet (UV) ink, or random colored fibers. Thesetechniques tend to be expensive, have a weak ground truth, andauthenticity detection relies on a manual decision by a recipient of theitem.

As an alternative, low cost surface structures have been exploited forcounterfeit detection by using their optical features. The randomness ofthe surface makes the structures physically unclonable or difficult toclone to deter duplications. Extrinsic surface structures can be createdby adding ingredients such as fiber, small plastic dots, air bubble,powders/glitters that are foreign to the surface. Intrinsic surfacestructures can also be intrinsic by exploring the optical effect of themicroscopic roughness of the surface, such as the paper surface formedby inter-twisted wood fibers.

The uniqueness of the inherent 3-D structure of the paper surface thatcan be exploited for authentication purposes is due to overlapped andinter-twisted wood fibers.

There are two fundamental types of reflection models: specular anddiffuse. The perceived intensity due to the mirror-like specularreflection is mainly dependent on the angle between the directions ofthe reflected light and eye/sensor, whereas the perceived intensity dueto diffuse reflection is mainly dependent on the angle between thedirections of incident light and the normal of the microscopic surface.Most surfaces are combinations of both surface types.

A piece of paper at different regions can have different dominantreflection types, but treating paper as a fully diffuse surface is oneoption. The majority of locations may follow this model, and theremaining locations can be considered outliers under this model.

FIG. 1 shows the surface normal direction and incident light directionof a particular spot in a microscopic view. The perceived intensity ofthe fully diffuse reflection model, l_(r)=λ·l·n^(T)v_(i), wherel∝cos^(K) θ and n^(T)v_(i)=cos φ, depends on the angle y between normaldirection of the surface at microscopic level, n=(n_(x), n_(y), n_(z)),and the direction where the incident light coming from, v_(i)=(v_(i,x),v_(v,y), v_(i,z)); the strength of the light at the current spot, l; andthe albedo, λ, characterizing the physical capability of reflecting thelight. In this discussion, λ is assumed to be constant over a wholepaper patch. Parameter l can be modelled in proportion to cos^(K) θ,where K is a positive number accounting for the effect of energyfall-off according to the inverse-square law, the effect offoreshortening, etc., and θ is the angle of incidence.

In the case of a scanner, θ is a factory specified design parameterrelating to the position of the linear light source and therefore fixedfor every pixel location; in the case of cameras, θs are generallydifferent for neighboring pixel locations.

SUMMARY

A method, according to certain embodiments, can include illuminating asurface of an item with a lighting source of a device. The method canalso include capturing a plurality of images of the surface by a cameraof the device during the illumination of the surface. The method canfurther include authenticating the item based on the plurality ofimages.

An apparatus, in certain embodiments, can include a light of a deviceconfigured to illuminate a surface of an item. The apparatus can alsoinclude a camera of the device configured to capture a plurality ofimages of the surface during the illumination of the surface. Theapparatus can further include a processor of the device configured toauthenticate the item based on the plurality of images.

According to certain embodiments, an apparatus can include means forilluminating a surface of an item with a lighting source of a device.The apparatus can also include means for capturing a plurality of imagesof the surface by a camera of the device during the illumination of thesurface. The apparatus can further include means for authenticating theitem based on the plurality of images.

BRIEF DESCRIPTION OF THE DRAWINGS

For proper understanding of the invention, reference should be made tothe accompanying drawings, wherein:

FIG. 1 shows the normal direction and incident light direction of aparticular spot in a microscopic view of a surface.

FIG. 2A illustrates the design of a registration container according tocertain embodiments.

FIG. 2B illustrates a tri-patch design of a registration containeraccording to certain embodiments.

FIG. 3 illustrates a block diagram of preparing an authenticatable item,according to certain embodiments of the present invention.

FIG. 4 illustrates a block diagram of authenticating an authenticatableitem, according to certain embodiments of the present invention.

FIG. 5 illustrates details of a norm map estimator, according to certainembodiments of the present invention.

FIG. 6 illustrates a system according to certain embodiments of thepresent invention.

FIG. 7 illustrates a method according to certain embodiments of thepresent invention.

FIG. 8 illustrates a further method according to certain embodiments.

DETAILED DESCRIPTION

FIG. 2A illustrates the design of a registration container according tocertain embodiments. This registration container can facilitate preciseregistration in experiments or in practical applications. Considering600 pixels per inch printing resolution, the container can be a squarebox of 400-by-400 in pixels, the line width can be 5 pixels, and therecan be four circles at the corners. A preliminary alignment based onfour boundaries can be achieved using a Hough transform, and subpixelresolution refinement with perspective transform compensation can thenbe carried out based on the circle markers. Lens location relative tothe captured surface in the world coordinate system can be readilycalculated from the estimated perspective transform matrix, and then thedirection of incident light at every pixel location can be known.

FIG. 2B illustrates a tri-patch design of a registration containeraccording to certain embodiments. The registration container in thisexample can include three of the designs illustrated in FIG. 2A, withthe additional incorporation of printed data, such as a quick response(QR) code in the central of the three patches. Alternatively or inaddition, the information could be in another form than a QR code andcould be located outside the patches or in another of the patches.

Optical characteristics of physically unclonable features (PUFs) can beused for verification. The PUF verification problem can be approached asan image authentication problem commonly formulated as hypothesis tests.The null hypothesis H₀ corresponds to incorrectly matched pairs of testand reference patches, whereas the alternative hypothesis H₁ correspondsto correctly matched pairs. The optimal decision rule maximizing thestatistical power is the likelihood-ratio test (LRT): rejects H₀ if

$\frac{f_{1}(x)}{f_{0}(x)} \geq \tau$holds, where x represents the test patch, ƒ₀ and ƒ₁ are the probabilitydensity functions under null and alternative hypotheses respectively,and τ is a threshold.

As a simple example, a hypothesis testing model differentiating a knownreference image w against all other images can be as follows:H ₀ : x=e ₀ , e ₀ ˜N(m1, Σ₀),H ₁ : x=w+e ₁ , e ₀ ˜N(0, σ₁ ² l).

Here, normally distributed e₀ stochastically represents any acquiredimage with a non-degenerate covariance matrix Σ₀ for image content andacquisition noise, 1 is an all 1 vector with the same dimension as x, mcorresponds to a value at the center of the linear range of the digitalrepresentation of luminance (i=128 for intensity in the range [0, 255]),w deterministically represents the reference image, and el is the imageacquisition noise (white Gaussian, with constant variance σ₁ ²). Whenthe patch x is represented by the x-component or y-component of thenormal vector field, the above under the hypothesis test setup is stillvalid with m=0. Sample correlation coefficient {circumflex over (ρ)}(w,x) against a threshold can be used as the decision rule, as in thisdiscussion.

Without knowing the exact direction of incident light, an estimate ofone component can be obtained as the difference between two scans inexactly opposite directions, canceling the effect of the unknownincident direction of the scanner light. See, for example, Clarkson etal. “Fingerprinting blank paper using commodity scanners,” in Proc. IEEESymposium on Security and Privacy, Berkeley, Calif., May 2009, pp.301-314.

Certain embodiments of the present invention address, for example, apaper authentication problem by exploiting optical features throughmobile imaging devices to characterize the unique, physically unclonableproperties of paper surface.

Prior work showing high matching accuracy either used a consumer-levelscanner for estimating a projected normal vector field of the surface ofthe paper as the feature for authentication, or used an industrialcamera with controlled lighting to obtain an appearance image of thesurface as the feature. Moreover, past explorations based on mobilecameras were very limited and have not had substantial success inobtaining consistent appearance images. One way to improve theauthentication performance is to use the intensity gradient basedfeatures of visually observable dots that are less sensitive to thechange of lighting, at the cost of increasing the design complexity ofthe authentication system.

Certain embodiments of the present invention recognize that the failureof past approaches with mobile cameras is due in part to theuncontrolled nature of the ambient light. More particularly, certainembodiments directly use images captured by mobile cameras forauthentication by exploiting the camera flashlight to create asemi-controlled lighting condition.

Certain embodiments also provide methods for estimating the microscopicnormal vector field of paper surface using multiple camera-capturedimages of different viewpoints. Thus, restricted imaging setups can berelaxed to enable paper authentication under a more casual, ubiquitoussetting of a mobile imaging device, which may facilitate duplicate orother counterfeit detection of paper documents and merchandisepackaging.

For example, certain embodiments focus on the intrinsic property of thepaper surface for counterfeit detection and deterrence. Optionally,however, the same techniques may also be applied to other kinds ofsurfaces beside paper surfaces, such as fabric surfaces, leathersurfaces, or any other surface with optically detectable microscopicsurface variation. Certain embodiments provide for a more casual,ubiquitous imaging setup using consumer-level mobile cameras undercommonly available lighting conditions.

As mentioned above, features based on intensity gradient of visuallyobservable dots may be less sensitive to the change of lighting and maybe used for authentication at the cost of higher algorithm complexityand moderate discrimination capabilities. Thus, certain embodiments mayincorporate such features in combination with the approaches describedherein. Furthermore, anti-duplication or anti-counterfeiting techniquesmay also or alternatively rely on other characteristics of a protecteditem.

Two aspects of a process may facilitate paper authentication via mobilecameras. First, the mobile captured images can be configured to becomparable in resolution and contrast to those captured by scanners.Second, lighting can be controlled to render a desirable imageappearance of the paper.

The first aspect can be qualitatively confirmed by comparing theacquired images from scanners and mobile cameras. Images acquired inboth ways do have significant intensity fluctuations within smallneighborhoods of pixels. The second aspect can be fulfilled byactivating the flash next to the camera lens on devices such as mobiledevices. As the relative position of the flash is fixed with respect tothe lens, the appearance of the surface can be reasonably expected for agiven position between the camera and the paper.

The use of camera flash can significantly improve the authenticationperformance of appearance images, and more importantly, can allow forthe estimation of the normal vector field in fine surface details. Byknowing the estimated location of the lens, the direction of incidentlight for every pixel of the paper can be calculated. Then, the normalvector of a particular pixel can be estimated by using the fully diffusereflection model, with a special treatment on the non-uniform intensityin camera images due to different distances from pixel locations to theflash. Thus, mobile camera-based techniques can obtain an effectiveestimate of the normal vector field of the paper surface to enableauthentication.

As uncontrolled light source(s) may be a major reason for lowauthentication performance using the appearance images as the feature, asemi-controlled lighting condition with the help of the built-inflashlight of mobile cameras can achieve proper authentication. Therelative positions among the light source, lens, and the paper patch canbe known, or at least can be estimated.

A simple case would be using the appearance of patches captured atlocations relatively fixed to the lens so that the effect of lighting isthe same. A more sophisticated case discussed below is to understand thephysics of lighting and with multiple appearance images to estimate thenormal vector field of the surface for authentication.

Because the image appearance is highly dependent on the camera's designparameters, such as relative positions of the lens and flash, as well asthe reflectivity or shadowing of the case, the approach may take intoaccount that the acquisition device at the user side cannot be limitedto a particular model.

In a simple case, for example, a reference image can be taken using adevice with a built-in camera and lighting source, such as an LED flashor other flash. The reference image can then compared to a subsequentlycaptured image, which can be captured under the same camera-surfacegeometry using the same type of device with the same physical layout ofcamera lens and lighting source. If a correlation between the images ishigh enough, then authenticity can be confirmed.

Because modern mobile cameras have improved in resolution in capturingfine details, it may be possible to estimate the normal vector field byusing multiple appearance images. This may be done if, for example,issues of camera geometry and lighting can be addressed.

Photometric stereo can be used to reconstruct surfaces using appearanceimages captured at different perspectives. However, the challenge hereis that the scale of interested surface is much smaller. The physicalmodel of light reflection can be appropriately selected and the lightingcan be controlled to exploit the possibility of obtaining meaningfulestimates of the normal vector field.

Given a generally flat paper surface and a typical mobile phone camera,there can be a gentle spatial intensity change at large scale, namely agentle macroscopic intensity change, with circular shaped level curves.This macroscopic intensity can be compensated to reveal the intensitychange due to the change of orientation of microscopic surface.

The macroscopic intensity may be proportional to the light strength atthe surface, l, and cosine of the incident angle, θ. We approximate themacroscopic intensity by the averaged perceived intensity of backgroundpixels over a small neighborhood N around a pixel location p:

${\overset{\_}{l_{r}}(p)} = {\frac{1}{{N(p)}}{\sum\limits_{k \in {N{(p)}}}{{\lambda \cdot {l(k)} \cdot {n(k)}^{T}}{v_{i}(k)}}}}$$(a) \approx {{\lambda \cdot {l(p)} \cdot \left\lbrack {\frac{1}{N(p)}{\sum{n(k)}}} \right\rbrack^{T}}{v_{i}(p)}}$(b) ≈ λ ⋅ l(p) ⋅ E[n(p)]^(T)v_(i)(p)(c) ≈ λ ⋅ l(p) ⋅ [0, 0, μ_(n_(z))]v_(i)(p) = λ ⋅ l(p) ⋅ μ_(n_(z)) ⋅ v_(i, z)(p)

where v_(i,z)(p)=cos θ at p, and where |N(p)| is number of pixels in thesmall neighborhood of p. Line (a) follows from the fact that l(k) andv_(i)(k) are approximately constant over a small neighborhood. Line (b)follows from ergodicity. Line (c) follows from the assumption thatnormal vectors are on average pointing straight up, E[n_(x)]=E[n_(y)]=0and E[n]=μ_(n) _(z) , where μ_(n) _(z) is a modeling constant between 0and 1.

For simplicity, median filtering can be applied over different shots.Thus, satisfactory estimation results for macroscopic intensity l_(r)can be obtained. With an estimated macroscopic intensity image l_(r) ,the normalized intensity, ζ(p), of an image at a particular location pcan be defined by compensating the macroscopic intensity, as below:

${\zeta(p)}\overset{def}{=}{{\frac{l_{r}(p)}{\overset{\_}{l_{r}}(p)} \cdot \mu_{n_{z}} \cdot {v_{i,z}(p)}} = {{n(p)}^{T}{v_{i}(p)}}}$

where n is the unknown normal vector to be estimated, μ_(n) _(z) is theunknown modeling constant, l_(r) is the image acquired under flashlight,and the terms v_(i), l_(r) , and v_(i,z) are already estimated.Normalized images can be obtained by dividing the original imagecaptured under flashlight by the macroscopic intensity image.

In order to quickly examine the correctness of modeling, parameterestimation can be carried out using handy off-the-shelf estimators suchas least-squares. To obtain meaningful estimates with least-squares, apaper patch can be captured at more than four different camera locationswith respect to the paper, where four is sum of the three unknownparameters of a normal vector and one image intensity offset parameterto be determined. The image intensity offset parameter can be aparameter that describes the offset of the image intensity. Capturingthe patch at 20 camera locations is one example, although other numbersof capture locations are also permitted.

The normal vectors at every pixel location can be estimated for a totalof 200×200 pixels. For each pixel location p, a system of linearequations can be set up for solving the normal vector with known orestimated quantities:

$\begin{bmatrix}\zeta_{1} \\\vdots \\\zeta_{M}\end{bmatrix} = {{\begin{bmatrix}v_{1} & 1 \\\vdots & \vdots \\v_{M} & 1\end{bmatrix}\begin{bmatrix}n_{x} \\\vdots \\b\end{bmatrix}} + \begin{bmatrix}e_{1} \\\vdots \\e_{M}\end{bmatrix}}$

where

$\quad\begin{bmatrix}\zeta_{1} \\\vdots \\\zeta_{M}\end{bmatrix}$can be referred to as ζ,

$\quad\begin{bmatrix}v_{1} & 1 \\\vdots & \vdots \\v_{M} & 1\end{bmatrix}$can be referred to as X,

$\quad\begin{bmatrix}n_{x} \\\vdots \\b\end{bmatrix}$can be referred to as β, and

$\quad\begin{bmatrix}e_{1} \\\vdots \\e_{M}\end{bmatrix}$can be referred to as e. The unknown parameter β can contain the normalvector and an intercept capturing any intensity bias, for example due toambient light, at location p. The observation vector ζ can consist ofnormalized intensity values at the collocated position p from images #1to #M. The data matrix X can be composed of vectors of incidentdirections. Noise from measurement and/or modeling can be modeled by thezero-mean error vector e.

The estimated normal vector field can give satisfactory authenticationperformance. Various factors, however, may affect the authenticationperformance.

In certain embodiments, twenty images can be used to estimate fourparameters in order to obtain good estimates with high confidence.However, even with merely five images, the authentication performancecan still be satisfactory in the sense that the sample correlationvalues may be significantly greater than 0 for correct matches. Thus,the number of images can be one such factor that can affect performance.

Another factor that can affect performance is the precision of theestimated lens location. The incident light direction v_(i) may have asignificant effect on obtained estimates for the normal direction field.In certain embodiments, v_(i) is itself an estimate from the perspectivetransform matrix that may be inaccurately estimated. When perturbationis in x- (or y-) direction, the x- (or y-) component of the estimatednormal vector field may have a reduction of about 0.15 in correlation,and the other component may have no change. When perturbation is indiagonal directions, both x- and y-components may have a reduction ofabout 0.1 in correlation. In spite of the reduction in correlation, thecorrect matches can still be perfectly separated from incorrect matches.Hence, the lens location estimation and the authentication performancemay not be significantly affected by a 10°-bias of the estimated lenslocation.

Certain embodiments may have various benefits and/or advantages. Forexample, according to certain embodiments it may be possible to usecameras and built-in flashlights of mobile devices to estimate thenormal vector field that is an intrinsic microscopic feature of thepaper surface for authentication purposes. Certain embodiments may,therefore, relax restricted imaging setup to enable paper authenticationunder a more casual, ubiquitous setting of a mobile imaging device,which may facilitate duplicate detection of paper documents andmerchandise packaging.

Certain embodiments may also be applicable to scenarios in which thecamera is not in parallel with the paper surface. This may pose achallenge due to the out-of-focus blur effect that may occur over partsof the paper surface.

FIG. 3 illustrates a block diagram of preparing an authenticatable item,according to certain embodiments of the present invention. As shown inFIG. 3, blank copy/cotton paper 310 can have a label ID 320 applied toit in a process 330 of printing an alignment box and quick response (QR)code or other auxiliary data, such as data related to label ID andproduct information. A QR code is just an example of a way to obtainauxiliary data, such as a label ID, to facilitate matching andauthentication.

This process 310 can produce a label 370 to be stuck to a package ordocument. More generally, the result of printing at 310 can provide asurface patch 340. At 350, there can be four scans from perpendicularscanning directions. A norm map can be estimated at 360 using thescanned images. This can be provided as a reference norm map 380.

FIG. 4 illustrates a block diagram of authenticating an authenticatableitem, according to certain embodiments of the present invention. Asshown in FIG. 4, at 405 there can be a testable label 405 on apackage/document. The user or user equipment can, at 410, take multipleshots from different perspectives using a mobile camera with a flash.This may generate multiple images 415 of the label. Image registrationcan occur at 420. This image registration at 420 can provide aperspective transform matrix 425 and aligned images 430. Surface patchimage extraction and QR image extraction or other auxiliary dataextraction can occur at 435, and can yield surface patch images 440 andQR image 450 or image of other auxiliary data. For example, certainembodiments are not limited to QR images, but can also apply to barcode, text, or any other form of auxiliary data.

A norm map estimator 445 can use mobile camera images. This estimator445 can rely on both the perspective transform matrix 425 and surfacepatch images 440. An output of the estimator 445 can be an estimatednorm map 480.

A QR decoder or other auxiliary data decoder 455 can decode the QR image(or other auxiliary data) to obtain a label ID 460. A reference norm mapretriever 465 can rely on the label ID 460 and a database 470 ofreference norm maps, to yield a reference norm map 485.

An authenticity decision block 490 can compare the estimated norm map480 to the reference norm map 485. Based on the comparison, theauthenticity decision block 490 can yield an authentic or not output495.

FIG. 5 illustrates details of a norm map estimator, according to certainembodiments of the present invention. As shown in FIG. 5, a norm mapestimator can receive a perspective transform matrix 510. A cameraposition estimator 520 can process the perspective transform matrix 510.The norm map estimator can also receive a surface patch image 530 andprocess it in a macroscopic intensity image estimator 540.

The norm map estimator can, at 550, calculate an incident angle forevery pixel. Using an output from 550 together with an output ofmacroscopic intensity image estimator 540 and surface patch image 530,the norm map estimator can, at 560, generate a normalized image. Thisgeneration step at 560 can be done on multiple images to yield multiplenormalized images, for example, at least four normalized images at 570.The normalized images 570 and an output of the calculation of incidentangles at 550 can be used to perform a norm map calculation at 580,thereby yielding an estimated norm map 590.

FIG. 6 illustrates a system according to certain embodiments of thepresent invention. As shown in FIG. 6, a system can include a labelgenerator 610 and a user equipment 620. Both devices can include atleast one processor 614, 624, at least one memory 615, 625, atransceiver 616, 626, and a camera or scanner 613, 623 (both a cameraand a scanner may be present, if desired). Optionally other suitableequipment can be substituted for camera/scanner 613, such as using analternative technology for mapping, such as microscopic analysis. Thelabel generator 610 may also include a printer 617. The printer 617 canbe configured to print a physical label, which may include a QR code orother metadata as described above. The user equipment 620 may include auser interface 621, which may be configured to provide an output of anauthentication result, as described above.

The processor(s) 614, 624 may be any suitable circuitry, chip, centralprocessing unit, or application specific integrated circuit. Thememory(ies) 615, 625 may be any storage mechanism, such as random accessmemory (RAM), read only memory (ROM), flash memory, solid state drivememory, electronically programmable memory, or the like. Theprocessor(s) 614, 624 and memory(ies) 615, 625 can be provided on a samechip as one another, or on different chips. The memory(ies) 615, 625 caninclude computer program instructions, such as computer program code, inany desired form, such as machine code, interpreted code, or the like.

The devices can be configured to use their transceiver 616, 626 tocommunicate across a network either with one another or with a third orother devices, such as an external database of reference norm maps, asmentioned above.

The user equipment 620 can further include a light 628, such as abuilt-in flash. For example, the user equipment 620 can be mobile cameraphone, with a built-in flash.

The devices of FIG. 6 may provide the means for executing any of themethods described herein. The devices can operate alone or incombination with one another.

FIG. 7 illustrates a method according to certain embodiments of thepresent invention. As shown in FIG. 7, a method can include, at 710,illuminating a surface of an item with a lighting source of a mobiledevice. The method can also include, at 720, capturing a plurality ofimages of the surface by a camera of the mobile device during theillumination of the surface. The method can further include, at 730,authenticating the item based on the plurality of images.

The method can additionally include, at 722, estimating a microscopicnormal vector field of the surface, wherein the authentication is basedon the estimate of the microscopic normal vector field.

The authentication can be based on physical characteristics of thesurface. For example, the authentication can be based on intrinsicphysical characteristics of the surface. More particularly, in certainembodiments, the authentication can be based on physically unclonableproperties of the surface.

The method can also include, at 724, estimating a location of a lens ofthe camera. The method can further include, at 726, calculating adirection of incident light for every pixel of an area of the surfacebased on the estimation. The authentication can be based on amicroscopic normal vector field of the surface determined based on thecalculated the direction of incident light. The method can additionallyinclude, at 728, weighting non-uniform intensity according to differentestimated distances from pixel locations to the lighting source whencalculating the direction of incident light.

The method can further include, at 740, providing an authenticationoutput to a user of the mobile device. The authentication can includedetecting that the item is a duplicate or counterfeit. Thus, the outputcan be a “pass” or a “warning,” “caution,” or “counterfeit detected,” orthe like.

FIG. 8 illustrates a further method according to certain embodiments. Asshown in FIG. 8, a method can include, at 810, illuminating a surface ofan item with a lighting source. This light source may be a dominatinglight source, such as a flash. Various flash illumination technologiesare permitted including, but not limited to, LED and incandescentillumination technologies.

The method can also include, at 820, capturing an image of the surfaceby a camera of a device during the illumination of the surface. Thecapturing can occur with a predefined camera-surface geometry. Thispredefined geometry can be determined using a viewfinder of the device,for example, aligning the camera's displayed picture edges with theedges of a reference box.

The method can further include, at 830, authenticating the item based oncomparison of an appearance of the captured image to a referenceappearance under the same predefined camera-surface geometry.Optionally, the reference appearance could be generated under adifferent camera-surface geometry but interpolated or otherwisetranslated to an appearance simulating the same predefinedcamera-surface geometry.

Although this example mentions a single image being captured andcompared, certain embodiments may rely on the capture of multiple imagesand the comparison of one or more images to multiple reference images.This embodiment can be used in combination with the above embodiments,or may be used alone, if desired.

The method of FIG. 7 or FIG. 8 may be implemented by, for example, theuser equipment 620 illustrated in FIG. 6. Furthermore, FIG. 4 can beconsidered a specific example of a general approach illustrated in FIG.7. Additionally, using an estimated surface norm map can be used alone,optionally the estimated surface norm map can be used in combinationwith an appearance-based approach involving compared a captured image toa reference image, depending on, for example, types of paper surfacesand applications.

When a camera's capturing resolution is high enough, the area covered byeach pixel may be relatively flat, and the normal vector assigned to thepixel may represent the physical surface direction of the area. Thecollection of the normal vectors therefore can serve as a fingerprintfor the paper surface. When the resolution is lower than theaforementioned scenario, however, the normal vector can still serve as ameaningful quantity. More specifically, the norm maps estimated from lowresolution images can be considered as a downsampled norm map using avirtual 2-D low-pass filter that relates the high and low resolutionimages.

Paper can be easily folded, resulting in a change of directions of thosesurfaces around the fold lines. In order to maintain a high correlationfor true matches, the following strategies can be applied. A firststrategy masks in correlation calculation those pixels whose surfacedirections are affected by folding. This method is intuitive but relieson the detection and segmentation of folded regions. As the distortionto the norm map field due to folding can be viewed as the addition of aslowly spatially varying trending surface, a second strategy is to applydetrending methods before calculating the correlation. For example,highpass filtering can be applied to remove the global trend. Such ahighpass filter may be designed to properly reject the frequencycomponents of the trending surface. Alternatively, parametric surfacescan be fitted to estimate the trending surface, and the resultingresidue can be used to perform correlation. A practical challenge maylie in the selection of a parametric surface that neither overfits norunderfits.

Perturbation analysis shows that the method according to certainembodiments is robust to inaccurate estimates of camera locations, andusing 6 to 8 images can achieve a matching accuracy of 10⁻⁴ in equalerror rate (EER). For example, in order to obtain an EER of 10⁻⁴, oneshould on average acquire at least six flash images if the correlationfollows a light tailed Gaussian distribution. In contrast, if thecorrelation deems to follow a heavy tailed Laplacian distribution, oneshould on average acquire at least eight flash images.

One having ordinary skill in the art will readily understand that theinvention as discussed above may be practiced with steps in a differentorder, and/or with hardware elements in configurations which aredifferent than those which are disclosed. Therefore, although theinvention has been described based upon these preferred embodiments, itwould be apparent to those of skill in the art that certainmodifications, variations, and alternative constructions would beapparent, while remaining within the spirit and scope of the invention.

We claim:
 1. A method, comprising: illuminating a surface of an itemwith a lighting source of a device; capturing a plurality of images ofthe surface by a camera of the device during the illumination of thesurface; authenticating the item based on the plurality of images;estimating a location of a lens of the camera; and calculating adirection of incident light for pixels of an area of the surface basedon the estimation, wherein the authentication is based on a microscopicnormal vector field of the surface determined based on the direction ofincident light.
 2. The method of claim 1, further comprising: estimatinga microscopic normal vector field of the surface, wherein theauthentication is based on the estimate of the microscopic normal vectorfield.
 3. The method of claim 1, wherein the authentication is based onphysical characteristics of the surface.
 4. The method of claim 3,wherein the authentication is based on physically unclonable propertiesof the surface.
 5. The method of claim 1, wherein the authentication isbased on comparing the appearance of the plurality of images toreference images.
 6. The method of claim 1, wherein the authenticationcomprises detecting that the item is a duplicate or counterfeit.
 7. Themethod of claim 1, further comprising: weighting non-uniform intensityaccording to different estimated distances from pixel locations to thelighting source when calculating the direction of incident light.
 8. Themethod of claim 1, further comprising: providing an authenticationoutput to a user of the device.
 9. An apparatus, comprising: a light ofa device configured to illuminate a surface of an item; a camera of thedevice configured to capture a plurality of images of the surface duringthe illumination of the surface; a processor of the device configured toauthenticate the item based on the plurality of images, estimate alocation of a lens of the camera; and calculate a direction of incidentlight for pixels of an area of the surface based on the estimation,wherein the authentication is based on the direction of incident light.10. The apparatus of claim 9, wherein the processor is furtherconfigured to estimate a microscopic normal vector field of the surface,wherein the authentication is based on the estimate of the microscopicnormal vector field.
 11. The apparatus of claim 9, wherein theauthentication is based on physical characteristics of the surface. 12.The apparatus of claim 11, wherein the authentication is based onphysically unclonable properties of the surface.
 13. The apparatus ofclaim 9, wherein the authentication comprises detecting that the item isa duplicate or counterfeit.
 14. The apparatus of claim 9, wherein thedevice comprises a mobile device.
 15. The method of claim 9, wherein theauthentication is based on comparing the appearance of the plurality ofimages to reference images.
 16. The apparatus of claim 9, wherein theprocessor is further configured to weight non-uniform intensityaccording to different estimated distances from pixel locations to thelighting source when authenticating the item.
 17. The apparatus of claim9, further comprising: a user interface configured to provide anauthentication output to a user of the device.
 18. An apparatus,comprising: means for illuminating a surface of an item with a lightingsource of a device; means for capturing a plurality of images of thesurface by a camera of the device during the illumination of thesurface; means for authenticating the item based on the plurality ofimages; means for estimating a location of a lens of the camera; andmeans for calculating a direction of incident light for pixels of anarea of the surface based on the estimation, wherein the authenticationis based on the direction of incident light.
 19. The apparatus of claim18, further comprising: means for estimating a microscopic normal vectorfield of the surface, wherein the authentication is based on theestimate of the microscopic normal vector field.
 20. The apparatus ofclaim 18, wherein the authentication is based on physicalcharacteristics of the surface.
 21. The apparatus of claim 20, whereinthe authentication is based on physically unclonable properties of thesurface.
 22. The apparatus of claim 18, wherein the authentication isbased on comparing the appearance of the plurality of images toreference images.
 23. The apparatus of claim 18, wherein theauthentication comprises detecting that the item is a duplicate orcounterfeit.
 24. The apparatus of claim 18, further comprising: meansfor weighting non-uniform intensity according to different estimateddistances from pixel locations to the lighting source when calculatingthe direction of incident light.
 25. The apparatus of claim 18, furthercomprising: means for providing an authentication output to a user ofthe device.
 26. A method, comprising: illuminating a surface of an itemwith a lighting source; capturing an image of the surface by a camera ofa device during the illumination of the surface, wherein the capturingoccurs with a predefined camera-surface geometry; authenticating theitem based on comparison of an appearance of the captured image to areference appearance under the same predefined camera-surface geometry;estimating a location of a lens of the camera; and calculating adirection of incident light for every pixel of an area of the surfacebased on the estimation, wherein the authentication is based on amicroscopic normal vector field of the surface determined based on thedirection of incident light.